Wireless cross-connect switch

ABSTRACT

Methods and systems for wireless packet switching include determining a schedule for a plurality of transceivers in an enclosure. The schedule specifies which of the plurality of transceivers will act as a transmitter and which will act as a receiver. Data is transmitted from each transmitter to a corresponding receiver with a configured beamforming direction. Data is transmitted from a transmitter to a corresponding receiver by a wired connection if an angle of the beamforming direction is lower than a minimum angle.

BACKGROUND Technical Field

The present invention relates to network communications and, moreparticularly, to wireless cross-connect switches for network clustersand data centers.

Description of the Related Art

Data centers show increased usage of large-bandwidth communicationsystems due to heightened server and storage consolidation, increasedvirtualization of system resources, and the design of multi-coreprocessor and multi-processor systems. The demand for interconnectionperformance increases proportionally with processor performance, asfaster processors need correspondingly faster input/output connections.However, interconnection technology scales at a much slower rate thanprocessor speed, resulting in an interconnect bottleneck. Conventionallythis is solved by increasing the number of communication links, but sucha solution causes problems of its own in data center management, as thenumber of cables for wired connections can rapidly become unmanageable.

In wired data centers, each server rack is connected to a network fabricof aggregation switches and core routers. There may be dozens of cablesbetween each rack and the switch/router fabric. The core router may havehundreds or thousands of ports and is generally not reconfigurable. Inaddition, such core routers are often expensive and power hungry. Thehundreds of cables passing to this single machine may impair data centercooling and occupy a substantial amount of space and data centerinfrastructure. While cables can be run along a drop ceiling to conservespace, the weight of the cabling often requires structural reinforcementto prevent the data center's ceiling from collapsing.

The problem is even more pronounced in all-to-all clusters. Someapplications work best on clusters where every node is directlyconnected to all other nodes. In an exemplary cluster having 32 nodes,where each node needs to connect to 31 others, a total of 496 cables areneeded. In a cluster of 128 processors, 8128 cables are needed. Thisplaces a large burden on the data support, logistically, economically,and structurally.

SUMMARY

A method for wireless packet switching include determining a schedulefor a plurality of transceivers in an enclosure. The schedule specifieswhich of the plurality of transceivers will act as a transmitter andwhich will act as a receiver. Data is transmitted from each transmitterto a corresponding receiver with a configured beamforming direction.Data is transmitted from a transmitter to a corresponding receiver by awired connection if an angle of the beamforming direction is lower thana minimum angle.

A system for wireless packet switching includes transceivers in anenclosure. An arbiter is configured to determine a schedule for theplurality of transceivers, wherein the schedule specifies which of theplurality of transceivers will act as a transmitter and which will actas a receiver. Each transmitter transmits to a corresponding receiverwith a configured beamforming direction. Each transmitter transmits datato a corresponding receiver by a wired connection if an angle of thebeamforming direction is lower than a minimum angle.

These and other features and advantages will become apparent from thefollowing detailed description of illustrative embodiments thereof,which is to be read in connection with the accompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

The disclosure will provide details in the following description ofpreferred embodiments with reference to the following figures wherein:

FIG. 1 is a diagram of a wireless network in accordance with the presentprinciples;

FIG. 2 is a block diagram of a wireless cross-connect switch inaccordance with the present principles;

FIG. 3 is a diagram of a wireless cross-connect enclosure in accordancewith the present principles;

FIG. 4 is a block/flow diagram of a method for operating a wirelesscross-connect switch in accordance with the present principles; and

FIG. 5 is a block/flow diagram of a method for operating a wirelesscross-connect enclosure in accordance with the present principles.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Embodiments of the present invention provide wireless cross-connectswitches that employ beamforming and beamsteering to create a flexibleand scalable interconnect system for server clusters and data centers.Two such switches are provided for herein: an inter-rack switch thatprovides switched wireless communication between individual racks, andan intra-rack switch that provides communication between devices withina single rack.

The present embodiments are disclosed as using, e.g., millimeter-wave(mmWave) transmissions, for example at 60 GHz or 120 GHz mmWavecommunications have high rates of absorption in many materials andattenuate quickly, such that it is difficult or impossible to eavesdropon mmWave communications without being in the line of sight. Usingfine-beam antenna arrays with beamsteering, communications can be madesecure and interference between transceivers can be minimized. mmWavecommunications also allow high data rates, such that mmWavecommunications can be compatibly used with fiber and copperinterconnects in hybrid systems. The wireless cross-connect switchesdescribed below provide for static and dynamic switching for packetswitched protocols and many be used for multicast, broadcast, andanycast addressing.

Although mmWave communications are therefore specifically contemplatedherein, it should be recognized that other forms of wirelesscommunication may be employed as an alternative. Due to the high carrierfrequency, there is considerable bandwidth available in mmWave, so thateven simple modulation schemes that can be implemented at very low powercan be used to carry high data rate signals. Since the antennas becomevery small at the high frequencies, the transmitters and receivers arecompact and can be built into dense arrays. In addition, techniques tosteer beams electronically can be used in conjunction with redundancy toimprove system-level reliability. Spatial multiplexing inherent tofan-beam antennas allows co-existence and smart re-use of the frequencyspectrum in more efficient way then traditional frequency divisionmultiplexing systems.

Referring now to the drawings in which like numerals represent the sameor similar elements and initially to FIG. 1, an exemplary data centerusing wireless switching is shown. The data center includes a set ofracks 102, each of which may have one or more connected devices such asservers. Each rack 102 has an mmWave transceiver 104 mounted on or in atop surface. On the ceiling of the data center is mounted an electroniccrosspoint switch 106 that includes a set of antennas 108 and an arbiter110.

The switch 106 is contemplated as having a hemispherical or otherwisecurved surface, with the antennas 108 being conformal antennas on thesurface of the switch 106 or on respective hemispherical protrusions.This geometry provides line-of-sight access to a greater number of racks102. The short wavelengths of mmWave communications allow very smallantennas to be used, whereas antennas for longer wavelengths would bemore difficult to implement conformally.

The transceivers 104 form mmWave radio links to and from the arbiterdevice 110 through the antennas 108. The antennas 108 are connectedwithin the switch 106 using wirelines, including, e.g., copper or fibercables. Each antenna 108 may represent a beamforming antenna array thatmay provide directional three-dimensional beamsteering directly to aspecific transceiver 104. The arbiter 110 performs scheduling,evaluating requests from different inputs and providing grants tospecific antenna/transceiver combinations. The arbiter 110 controls theantennas 108 to specify beam directions and, in doing so, can flexiblyreconfigure the switch 106 as needed.

The switch 106 may further function as an aggregator, collecting andmultiplexing communications from the racks 102 for transmission to otherclusters. In such a system, there may be other switches 106 servingother racks 102 in the data center, and communication between theseclusters may be managed by a core router located, for example, in theceiling. Communications between the switches 106 and the core router maybe accomplished through fiber, coaxial, or wireless interconnects.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc. or any suitable combination ofthe foregoing. Computer program code for carrying out operations foraspects of the present invention may be written in any combination ofone or more programming languages, including an object orientedprogramming language such as Java, Smalltalk, C++ or the like andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The program codemay execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through any type of network, includinga local area network (LAN) or a wide area network (WAN), or theconnection may be made to an external computer (for example, through theInternet using an Internet Service Provider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks. The computer program instructions may also beloaded onto a computer, other programmable data processing apparatus, orother devices to cause a series of operational steps to be performed onthe computer, other programmable apparatus or other devices to produce acomputer implemented process such that the instructions which execute onthe computer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblocks may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

Reference in the specification to “one embodiment” or “an embodiment” ofthe present principles, as well as other variations thereof, means thata particular feature, structure, characteristic, and so forth describedin connection with the embodiment is included in at least one embodimentof the present principles. Thus, the appearances of the phrase “in oneembodiment” or “in an embodiment”, as well any other variations,appearing in various places throughout the specification are notnecessarily all referring to the same embodiment.

It is to be appreciated that the use of any of the following “/”,“and/or”, and “at least one of”, for example, in the cases of “A/B”, “Aand/or B” and “at least one of A and B”, is intended to encompass theselection of the first listed option (A) only, or the selection of thesecond listed option (B) only, or the selection of both options (A andB). As a further example, in the cases of “A, B, and/or C” and “at leastone of A, B, and C”, such phrasing is intended to encompass theselection of the first listed option (A) only, or the selection of thesecond listed option (B) only, or the selection of the third listedoption (C) only, or the selection of the first and the second listedoptions (A and B) only, or the selection of the first and third listedoptions (A and C) only, or the selection of the second and third listedoptions (B and C) only, or the selection of all three options (A and Band C). This may be extended, as readily apparent by one of ordinaryskill in this and related arts, for as many items listed.

Referring now to FIG. 2, a block diagram of the switch 106 is shown. Aset of antennas 108 connect to respective transceivers 204 which aretuned to receive and transmit mmWave communications. A framer 206inserts and extracts characters to indicate the start of header and endof transmission portions of packets. A port controller 208 processesframed data that is multiplexed/demultiplexed in serializer/deserializer210. A cross-connect 212 forms connections between paths that receive asignal to paths that transmit a signal, thereby allowing signals fromany arbitrary antenna 108 to be retransmitted on any arbitrary secondantenna 108. The cross-connect 212 may be cascaded to provide switchingfor larger configurations. For example, sixteen 64×64 cross-bar modulescan be connected to form a 256×256 cross-bar. An arbiter and crosscontrol module 214 receives information from theserializer/deserializers 210 and controls the cross-connect 212accordingly. The arbiter 214 may furthermore be in communication withthe arbiters of other switches 106, connected wirelessly or by a wiredconnection to a core router.

The arbiter 214 controls data flow between the inputs and the outputs ofthe cross-connect switch 212. The arbiter 214 looks at the segmenteddata from port controller and evaluates requests from different inputs.The requests of different inputs include a specific address of acommunicated device located in a different server. The arbiter 214 thenevaluates the priority, provides grants, and reconfigures the switch 212to allow routing of the packet to the right server and communicationdevice.

The port control 208 is a network processor that is used for thepolicing and scheduling of data. The policy of the port controller 208can change from time to time and can be a function of differentnetworking scenarios or changes in the network. For example, although agiven channel may need to communicate with some servers, it may beprevented from doing so because a policy has changed. Schedulingincludes timing information and a list of addresses a specific packetshould be sent to.

The serializer/deserializer 210 multiplexes and demultiplexes both thedata as well as control information. Fast synchronization is possibleafter switch reconfiguration using data clock recovery afterreconfiguring the cross-connect switch 212. Such synchronizationincludes the alignment of frames received from port controllers 208.Alignment of frames received from the port controllers 208 uses smallfirst-in-first-out buffers to ensure frame synchronization.

The arbiter and cross-control 214 includes a tag evaluation module thatprovides requests for reconfiguration. A tag is a portion of theincoming data that includes a packet address and some measure ofpriority related to a packet, allowing the arbiter 214 to determinewhere the packet should go and how quickly it needs to go there.

On the physical layer, the arbiter and cross-control 214 may extract thetags, or headers, based on precise timing information. For example,after receiving an open header, the next m bits may refer to thetype/protocol, followed by n bits that refer to the source address,followed by p bits that refer to the destination address, and so on.This information is then communicated to the port controllers 208 toconfigure the connection based on a given routing policy.

Arbitration depends largely on the routing policy (e.g., source routing,in which case the entire path of a packet is precomputed by the sourcenode and is stored, for example, in a lookup table). In the simplestcase, when there is no conflict, input and output transceivers 204 areconfigured by the arbiter 214, which matches free network resources withthe packets that request those resources. If there is a conflict (e.g.packets from two or more input ports that simultaneously request thesame output port), then the arbiter 214 resolves the conflict (forexample, based on priority tags, or by round-robin allocation, or bydropping packets if permitted).

Referring now to FIG. 3, a wireless intra-rack switch 300 is shown. Theswitch 300 occupies a cylindrical Faraday cage 302 and includesdirectional transmitters 304 and omni-directional receivers 306. FIG. 3shows a single cross-section of the switch 300, and it should beunderstood that multiple such cross-sections may be stacked to increasethe switch's capacity. The transmitters 304 employ beam forming todirect transmissions to specific receivers 306 and may operate at, forexample, mmWave frequencies. The Faraday cage 302 prevents transmissionsfrom leaking out of the switch 300, thereby preventing eavesdropping andenhancing security. Although it is specifically contemplated that theFaraday cage 302 is formed from a conductive material, the cage 302 maybe replaced with a layer of material that is absorptive to mmWavefrequencies, such as foam. The individual transmitters 304 and receivers306 may furthermore be connected to an arbiter 308 via a wirelessconnection or by a wired interconnect.

It should be noted that each of the transmitters 304 and receivers 306may actually be part of a bi-directional communication structure. Thetransmitter 304 and receiver 306 may alternate sending and receivingroles. The arbiter 308 controls each of the devices and coordinateswhich devices will act as transmitters 304 and which will act asreceivers 306 for a given transmission period. This may be reorganizedperiodically or as needed.

Although the switch in FIG. 3 is shown as having eighttransmitters/receivers, the number of different devices may beincreased, limited only by how small a beam angle the transmitters 304can produce. Practical values for one embodiment may be, e.g., 5-10°.The switch 300 may furthermore be used in a broadcast mode, where asingle transmitter 304 transmits to all receivers 306. In this case, thetransmitters 304 are omni-directional as well as the receivers 306.Multicast is also possible, where one transmitter 304 communicates witha group of receivers 306. This may be accomplished by connecting with asingle receiver 306 and having that receiver 306 share the data with itsneighbors via a wired connection. Such a wired connection provides forcommunication between devices that may be too close to one another to bereached by beamforming.

In addition to the infrastructural applications shown above for datacenters, other applications are envisioned. For example, wirelesslyswitched memory may use a wireless switch 300 as shown above tofacilitate communication between a processor cluster and a memorycluster, with dynamic processing of the memory as called for by theworkload. Memory and processing nodes can be separated by a substantialdistance (e.g., 10 m), allowing a first processor to access memoryblocks from another processor.

Referring now to FIG. 4, a method for operating a wireless cross-connectswitch 106 is shown. Block 402 receives a packet from a first device ata first transceiver 204 with an antenna 202. The first transceiver 204is configured to communicate with the first device using a directionalbeam to minimize interference from other devices. A port controller 208analyzes the packet and communicated with the arbiter 214 to determine adestination device for the packet at block 404. At block 406 the arbiter214 controls a cross-connect 212 to connect the packet to a secondtransceiver that can reach the destination device. Block 408 configuresan antenna 202 of the second transceiver 204 to form a beam directed atthe destination device to allow transmission of the packet.

Referring now to FIG. 5, a method for operating a wireless switchingenclosure 300 is shown. At block 502, an arbiter 308 collectsinformation from one or more first transceivers 304 that they haveinformation to communicate to one or more corresponding secondtransceiver 306 in the enclosure 300. The arbiter 308 determines aschedule of transmissions in accordance with information provided by theport controller in block 504. The schedule includes designations for thetransceivers to determine which transceivers will operate astransmitters 304 and which will operate as receivers 306 for theduration of a transmission period. The transmission period may be apredetermined length of time or may be responsive to network conditionsby, e.g., ending when a queue reaches a threshold number of packets thatcannot be transmitted to their destination in the current schedule.

The arbiter 308 then controls the first transceiver 304 and the secondtransceiver 306 to coordinate the transmission in block 506, such thatthe first transceiver 304 emits a directed beam to the secondtransceiver 306 operating as a receiver. The beam may be directed to areceiver 306 that is vertically as well as radially displaced from thetransmitter 304. The receiver 306 may also be configured according tosome beamforming direction or may simply be configured to receiveomnidirectionally.

Block 508 transmits packets from transmitters 304 to their correspondingreceivers 306 until the end of the transmission period. At this point,processing returns to block 502 to determine a new schedule accordingcurrent network conditions and switching needs.

Having described preferred embodiments of a wireless cross-connectswitch methods (which are intended to be illustrative and not limiting),it is noted that modifications and variations can be made by personsskilled in the art in light of the above teachings. It is therefore tobe understood that changes may be made in the particular embodimentsdisclosed which are within the scope of the invention as outlined by theappended claims. Having thus described aspects of the invention, withthe details and particularity required by the patent laws, what isclaimed and desired protected by Letters Patent is set forth in theappended claims.

What is claimed is:
 1. A method for wireless packet switching,comprising: determining a schedule for a plurality of transceivers in anenclosure, wherein the schedule specifies which of the plurality oftransceivers will act as a transmitter and which will act as a receiver;transmitting data from each transmitter to a corresponding receiver witha configured beamforming direction; and transmitting data from atransmitter to a corresponding receiver by a wired connection if anangle of the beamforming direction is lower than a minimum angle.
 2. Themethod of claim 1, further comprising repeating said steps ofdetermining and transmitting data for subsequent transmission periods.3. The method of claim 2, wherein each transmission period ends when atransmission queue accumulates a threshold number of packets that cannotreach a specified destination under the determined schedule.
 4. Themethod of claim 1, wherein the plurality of transceivers are inwardlyfacing within the enclosure.
 5. The method of claim 1, wherein theplurality of transceivers are arranged in a plurality of verticallyarranged rings.
 6. The method of claim 5, wherein the beamformingdirection selects a radial direction and a vertical direction.
 7. Themethod of claim 1, wherein determining the schedule comprises analyzingpackets queued by the computing devices to determine destinationdevices.
 8. The method of claim 1, wherein one or more transmitters areconfigured with a beamforming pattern that is directed to multiplereceivers.
 9. A system for wireless packet switching, comprising: aplurality of transceivers in an enclosure; and an arbiter configured todetermine a schedule for the plurality of transceivers, wherein theschedule specifies which of the plurality of transceivers will act as atransmitter and which will act as a receiver, wherein each transmittertransmits to a corresponding receiver with a configured beamformingdirection, and wherein each transmitter transmits data to acorresponding receiver by a wired connection if an angle of thebeamforming direction is lower than a minimum angle.
 10. The system ofclaim 9, wherein the plurality of transceivers and the arbiter arefurther configured to repeat said steps of determining and transmittingdata for subsequent transmission periods.
 11. The system of claim 10,wherein each transmission period ends when a transmission queueaccumulates a threshold number of packets that cannot reach a specifieddestination under the determined schedule.
 12. The system of claim 9,wherein the plurality of transceivers are inwardly facing within theenclosure.
 13. The system of claim 9, wherein the plurality oftransceivers are arranged in a plurality of vertically arranged rings.14. The system of claim 13, wherein the beamforming direction selects aradial direction and a vertical direction.
 15. The system of claim 9,wherein the arbiter is further configured to analyze packets queued bythe computing devices to determine destination devices.
 16. The systemof claim 9, wherein one or more transmitters are configured with abeamforming pattern that is directed to multiple receivers.